Heretofore, learning, memorization, identification, etc. are discussed in "Parallel Distributed Processing I and II" by Mcclelland and Rumelhart (MIT Press, 1986). However neither knowledge on the cerebral physiology of living body, which is developed in the highest degree, is reflected therein nor discussion is done on the structure of the network, the speed of calculation, etc., which are problems, in the case where a practical application thereof is premised. In addition, no method for constructing the network for an object depending on the time is described therein.
On the other hand, a method for solving a neural network as an energy minimizing method is described in "Hop-field & Tank" (Science, Vol. 233 pp. 625-633 (1986)). However the neural network dealt with there is restricted to a monolayer and any solution cannot be obtained within a practical calculation time.
Hereinbelow a conventional technique by the minimum and maximum searching method for solving the neural network as an energy minimizing problem will be explained.
When the minimum (maximum) of a given cost function E was obtained, in the case where the cost function had a number of extreme values, generally it was difficult to obtain this minimum by the definite hill-climbing method as a conventional method. This is because, when a value in the neighborhood of a certain extreme value is given as an initial value, the system falls in a minimum value close thereto because of the fact that the method is definite and it is not possible to get out therefrom. Heretofore, in order to solve this problem, a definite hill-climbing method called simulated annealing has been proposed. Simply speaking, it is tried to reach the final destination by making it possible not only to climb the mountain but also to descend therefrom with a certain probability. By the method most widely utilized, taking a problem for obtaining the smallest value of E as an example, it can be solved as follows. At first, instead of considering directly the cost function E, it is considered to maximize a Bolzmann distribution P-exp (-E/T). The parameter T introduced therein is called temperature, which is introduced in order to generate random noise to make it possible to treat the problem statistically. Consequently, when the value obtained by calculation reaches a minimum value, it is necessary to set T at 0 and to make it stay at the minimum value without error. It is the greatest problem of the simulated annealing to determine the cooling schedule how to decrease T.
As discussed in IEEE Transaction on Pattern Analysis and Machine Intelligence, vol. 6, pp. 720-741, (1985), by the Geman brothers' schedule widely utilized heretofore, states are generated according to the Bolzmann distribution to fulfil T(t)=T.sub.0 /log (t+1), to being a positive constant. Here t corresponds to the number of Monte Carlo simulations and here it is defined that it represents the time. It is a matter of course that as t increases, T(t) approaches 0. Although several examples, in which this method can be successfully applied, have been already reported, there are many cases where it is not always successfully applied. Further, as discussed recently by Szu and Hartley in Physics Letters, vol. 123, pp. 157-161, (1987), in order to increase the convergence to the maximum of P, another schedule of T(t)=T.sub.0 /t+1 has been proposed, which uses Lorenz distribution having a wider spread in stead of Bolzmann distribution. However a disadvantage common to these schedules is that no function form of the cost function, which is to be minimized, is taken into account at all. It is not reflected on T(t) what kind of cost barriers (difference in the cost between a minimum value and a maximum value in the neighborhood thereof) is to be climbed and when the final value is reached (when T is set at 0). Numerically it is proved that the desired greatest value of P is always reached by both the methods, when infinite time has lapsed. However, in practice, although there are cases where the smallest value is reached within a finite time, during which the simulation can be executed, since there are many cases where it is not, the value of utilizing them is not always high.
The disadvantage common to the schedules stated above is that the function form of the cost function, which is to be minimized, is not reflected on the temperature T. Therefore, in practice, the smallest value cannot be obtained often within a finite time, during which a simulation can be executed.